Learning software assisted object joining

ABSTRACT

Systems and methods for automated manufacture are provided. Nominal data measurements are obtained for an article. An identification scan is performed of parts within the work area by a machine vision system. An initial scan of the parts within or adjacent to the work area identified as being needed to form said article is performed by the machine vision system to identify target points. The target points are compared at a controller with the nominal data measurements. Automated material handling machines are commanded to grasp and move the parts within said work area to form the article.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. application Ser. No.17/348,403 filed Jun. 15, 2021 (the “'403 Application”), which is acontinuation-in-part of U.S. application Ser. No. 17/078,611 filed Oct.23, 2020 (the “'611 Application”), which is a continuation-in-part ofU.S. application Ser. No. 16/664,443 filed Oct. 25, 2019 (the “'443Application”), which claims the benefit of U.S. Provisional ApplicationSer. No. 62/751,014 filed Oct. 26, 2018. The '611 Application is also acontinuation-in-part of the '443 Application. This application is also acontinuation-in-part of the '611 Application. The disclosures of each ofthe foregoing are hereby incorporated by reference as if fully restatedherein.

TECHNICAL FIELD

Exemplary embodiments relate generally to systems and methods forlearning software assisted object joining, such as in a fixturelessmanner.

BACKGROUND AND SUMMARY OF THE INVENTION

Dedicated hardware fixtures are often used to secure and locate sheetmetal parts for welding. Such sheet metal parts may be welded togetherto create subassemblies, which may subsequently be used to make a largerproduct. A common example of where such processes are utilized is theautomobile manufacturing industry. Regardless, a unique fixture mustgenerally be designed and manufactured to accommodate each subassembly.While some so-called flexible fixture systems are available, the costsof designing and manufacturing such flexible fixture systems aresignificant due to the complexity of design required to accommodate evenminor changes. Furthermore, even such flexible fixture systems may bequickly rendered obsolete upon the introduction of product changes. Forexample, without limitation, in the automobile industry, such fixturesystems may need updating with each model or production line change. Theaverage costs for fixturing in an automotive assembly plant is estimatedto be $150-300 million every 3-4 years. The use of robots and otherautomated assembly systems to perform material handling and joining areknown. For example, many Integrators of robots and automated assemblysystems are available in various industries. As another example, robotsand automated assembly systems using fixtures to provide physical datumpoints are known, such as are described in U.S. Pat. No. 10,095,214issued Oct. 9, 2018 and U.S. Pat. No. 10,101,725 issued Oct. 16, 2018.While the use of such robots and automated assembly systems may reducecertain expenses and provide improved consistency in manufacturing, whatis needed is the ability to utilize learning A.I. software to replacefixtures and physical datums with virtual datums.

In accordance with the present invention, a reconfigurable, fixturelessmanufacturing system and method assisted by learning A.I. software isprovided. One or more material handling robots may be provided at anassembly area. The assembly area may be located within a manufacturingfacility, for example without limitation. One or more joining robots maybe provided at the assembly area. Each of the material handling robotsmay be configured to selectively engage any of a number of materialhandling devices. Each of the material handling devices may comprise oneor more gripping elements, grippers, clamps, some combination thereof,or the like. Each of the material handling devices may be configured tograsp a number of differently shaped parts. In exemplary embodiments,such parts are grasped by inserting a first gripping element of a givenmaterial handling device into a locating hole on the part and pressing asecond and third gripping element against walls of the part, preferablyon opposing sides of the locating hole.

A given material handling robot may grasp a given part with a givenmaterial handling device and may move the part to a particular locationwithin the assembly area where the material handling device may bereplaced from the material handling robot in order to accommodate adifferent part's family sizes and shapes. In other exemplaryembodiments, the material handling robot may instead hold the materialhandling device at the particular location within the assembly area. Inexemplary embodiments, a number of parts may be stacked or otherwiseadjoined to one another to form a subassembly within the assembly area.Such parts may be stacked or otherwise adjoined at a docking station.The docking station may be placed atop an autonomous guide vehicle (AGV)or mounted to the floor.

A machine vision system may perform an alignment scan of thesubassembly. The machine vision system may be configured to capture thelocation of selected datums of the subassembly. The datums may beparticular features of or locations on the subassembly. The machinevision system may transmit the location of such datums to a controller.The controller may compare the location of the scanned datums withpredetermined locations to determine a best fit for the parts to createthe subassembly stored at the learning A.I. software. If needed, theparts may be adjusted spatially. The process for determining the bestfit and providing spatial adjustment may be as shown and described inU.S. Pat. No. 10,095,214 issued Oct. 9, 2018 and U.S. Pat. No.10,101,725 issued Oct. 16, 2018, which are hereby incorporated byreference herein in their entireties. The one or more joining robots mayjoin the parts, such as by welding, fastening, or riveting, somecombination thereof, or the like to complete the subassembly. Thematerial handling device and/or the material handling robots may bedisabled from movement during the joining process to apply breakingeffect, other methods can also be utilized to apply breaking to holdparts in position while joining. The machine vision system may performan inspection scan of the completed subassembly. Any discrepanciesbetween the inspection scan and the alignment scan may be transmitted tothe learning A.I. software by way of the controller. Additional methodsinclude embedding the best fit A.I. algorithms directly in the cameraprocessor or in the robot controller software to minimize and eliminateadditional hardware and cabling. The learning A.I. software may beconfigured to adjust the stored datums to compensate for suchdiscrepancies when producing the next subassembly. This machine learningprocess may permit optimization of the assembly process through multipleproduction iterations of a given subassembly. The flexibility of thematerial handling devices and the use of the learning A.I. software mayprovide the ability to use the same, or a substantially similar, systemto handle and join a number of differently shaped, sized, arranged, orthe like, parts in a number of different orientations to produce anumber of differently shaped, sized, arranged, or the like,subassemblies or assemblies which may be improved through eachmanufacturing iteration. Furthermore, the use of material handlingrobots and material handling devices to secure the parts may provide afixtureless assembly process. By storing the virtual datums, the needfor a physical fixture to provide physical datum points may beeliminated or significantly reduced.

In exemplary embodiments, one or more components of the machine visionsystem may be configured to determine if a worker or other individual iswithin the assembly area. Movement of the material handling robots maybe prevented or halted while the individual is within the assembly area.The material handling robots may be configured to grasp one or moreparts and move them into an assembly position for an inspection scan bethe machine vision system to verify that certain features are presentand properly aligned. In such embodiments, the reconfigurablemanufacturing systems and methods may operate for part inspection andverification and joining need not necessarily be performed, thoughsubsequent joining is contemplated.

Other types and kinds of handling may be desirable. For example, it maybe desirable to pick up a particular part from among a number of partsand place it in a new location. The new location may be adjacent toother parts to form part of all of a subassembly or assembly, such asbut not limited to, in a docking device, as held by other robots, on atable or other surface, or the like. This is particularly common in amass manufacturing process, such as an assembly line where parts arecontinuously made to make larger articles. However, two subsequent partsare generally not identical. They may be shifted in orientation whenpresented to a robot, be of various sizes or shapes (usually within agiven tolerance), or the like. This may lead to difficulty in joiningthe two or more parts in a way which preserves the overall intendeddesign of the subassembly or assembly. Therefore, what is needed islearning software assisted, fixtureless object pickup and placementsystems and methods.

Systems and methods for learning software assisted, fixtureless objectpickup and placement are provided. A workpiece table may be stored at acontroller. The workpiece table comprises a list of parts, such as bypart identifier, which form an assembly. An assembly table may be storedat the controller. The assembly table comprises a list of desiredassemblies, such as by assembly identifier. A target points table may bestored at the controller. The target points table may comprise one ormore desired target points for each part. The target point table mayinitially be populated with actual measurement data, such as obtained bythe machine vision system of each actual part and/or the location wherethe part is to be finally placed. For example, measurements may be madeof a docking station for the part or of the part to which the part beinghandled is to be joined. The target points may reflect actual or virtualdatums on the part. Each target point may be weighted to reflecttolerances desired between the parts of the assembly. Each part may beassociated with multiple target points, which may reflect surfaces orfeatures of the part and/or desired locations of the same to form theassemblies. The desired target points may be selected relative to oneanother to fit the various parts into the desired subassembly orassembly. The desired target points may be determined from a scan of areference, idealized part.

One or more computers may designate an origin at a common referenceframe. The computer(s) may represent coordinates of a docking station asvariables in the target point table. The computer(s) may express allmeasured coordinates in the common reference frame. The computer(s) mayconstruct one or more matrices of ordered pairs to represent each pairof mating target points of the actual measured locations of the targetpoints and final, desired target point locations for each part and storethese ordered pairs as calculated coordinates in the target point table.The initial, measured target point locations may be determined by a scanof one or more parts by a machine vision system. The desired targetpoint locations may be pre-programmed or may be provided with referenceto an idealized part.

Assuming the actual, measured locations do not align with, or are notwithin a predetermined range of, the desired target point locations, thecontroller may utilize an iterative, learning software algorithm todetermine a best fit solution for movement of the part(s) to be withinpredetermined range of the desired target points. The best fit solutionmay be determined by applying vectors in virtual space between theactual measured target points and the desired target points of the part.The algorithm may be configured to prioritize the solution based on theweights associated with each target point. As such, the algorithm mayselect the solution among the calculated potential solution sets whichminimalizes the need to shift the parts but fits the desired targetpoints with priority given to higher weighted target points, such thatthose higher weighted target points are associated with the smallestvectors. Upon finding a best fit, the controller may determine thepositions of all material handling robots. The controller may then causethe material handling robots to pick up and place each part to form theassembly. One or more steps may be repeated, as needed, to form the sameor different assemblies.

For example, without limitation, a second set of parts may be receivedfor a second assembly where the exact position of the parts may differfrom those used to create a first assembly. The machine vision softwaremay determine the location of various target points for the second setof parts and may determine how the material handling robots should shiftor be adjusted to grasp the repositioned parts. This may be realized by,for example without limitation, by determining vectors between thedesired position of each target point and the measured position of saidtarget points. A solution which minimizes the overall length of all suchvectors may be selected while still giving priority to higher weightedtarget points such that the vectors associated with such higher weightedtarget points are smaller or smallest.

The systems and method shown and/or described herein may be used withany number or type of fixturing systems, including but not limited to,entirely fixtureless systems, entirely fixtured systems, and hybridsystems (part fixtured, part fixtureless). Robotic vehicles may beutilized with some or all of the components of the systems, such as butnot limited to, for transporting parts, subassemblies, or assembliesto/from a work area, on or separate from, fixated or fixturelesssystems. The systems shown and/or described herein may be utilized withany type of kind of manufacturing processes, including but not limitedto, material handling and placement, welding, adhesion, fastening,unfastening, inspection, combinations thereof, and the like. Theartificial intelligence adaptation systems and methods shown and/ordescribed herein may be used with any applications shown and/ordescribed herein, such as but not limited to, for fastening tocompensate for over or under torquing, placement, spring back,combinations thereof, or the like. Such AI adaptation may be performedon a priority basis, such as to prioritize particular feature targetpoints by adaptive weightings.

Further features and advantages of the systems and methods disclosedherein, as well as the structure and operation of various aspects of thepresent disclosure, are described in detail below with reference to theaccompanying figures.

BRIEF DESCRIPTION OF THE DRAWINGS

In addition to the features mentioned above, other aspects of thepresent invention will be readily apparent from the followingdescriptions of the drawings and exemplary embodiments, wherein likereference numerals across the several views refer to identical orequivalent features, and wherein:

FIG. 1 is a perspective view of an exemplary A.I. drivenreconfigurable/fixtureless manufacturing system of the presentinvention;

FIG. 2 is a flow chart with exemplary logic for use with the system ofFIG. 1;

FIG. 3 is a perspective view of an exemplary gripping element of thepresent invention for use with the system of FIG. 1;

FIG. 4 is a perspective view of an exemplary material handling devicefor use with the system of FIG. 1, also indicating detail A;

FIG. 5 is a detailed perspective view of detail A of FIG. 4 with thepart rendered translucent so that additional components of the materialhandling device are visible;

FIG. 6A is a top view of a subassembly and material handling devices ofthe system of FIG. 1 shown in isolation, also indicating detail B;

FIG. 6B is a perspective view of a portion of FIG. 6A;

FIG. 6C is another perspective view of a portion of FIG. 6A;

FIG. 6D is another perspective view of a portion of FIG. 6A;

FIG. 7 is a perspective view of the system of FIG. 1 with certainmaterial handling robots not illustrated so that the subassembly andmaterial handling devices can be seen more clearly;

FIG. 8 is a detailed bottom view of detail B of FIG. 6A;

FIG. 9 is a perspective view of the AI driven reconfigurablemanufacturing system of FIG. 7 also illustrating an exemplary machinevision system in use therewith;

FIG. 10 is a detailed perspective view of FIG. 8 with a joining robot inuse;

FIG. 11 is a perspective view of a completed subassembly undergoing aninspection scan and an exemplary AGV;

FIG. 12 is an exemplary flow diagram for the A.I. driven reconfigurablemanufacturing process of the present invention;

FIG. 13 is a perspective view of another exemplary AI drivenreconfigurable manufacturing system of the present invention with A.I.learning;

FIG. 14 is another perspective view of the system of FIG. 13;

FIG. 15 is a detailed perspective view of the system of FIG. 13;

FIG. 16 another detailed perspective view of the system of FIG. 13;

FIG. 17 another detailed perspective view of the system of FIG. 13;

FIG. 18 is a flowchart with exemplary logic for operating the system ofFIG. 13 in accordance with the present invention;

FIG. 19 is a simplified system diagram for an exemplary pickup andplacement system in accordance with the present invention;

FIG. 20 is a top view of exemplary components of the system of FIG. 19;

FIG. 21 is a simplified system diagram for the system of FIG. 19 withinan exemplary manufacturing facility;

FIG. 22 is a more detailed system diagram for the system of FIG. 21;

FIG. 23A is an exemplary material handling portion of an exemplarymaterials handling robot for use with the system of FIG. 19;

FIG. 23B is an exemplary material handling portion of an exemplarymaterials handling robot for use with the system of FIG. 19;

FIG. 23C is an exemplary material handling portion of an exemplarymaterials handling robot for use with the system of FIG. 19;

FIG. 23D is an exemplary material handling portion of an exemplarymaterials handling robot for use with the system of FIG. 19;

FIG. 23E is an exemplary material handling portion of an exemplarymaterials handling robot for use with the system of FIG. 19;

FIG. 23F is an exemplary material handling portion of an exemplarymaterials handling robot for use with the system of FIG. 19;

FIG. 24 is a plan view of exemplary parts, subassemblies, and assembliesfor use with the system of FIG. 19;

FIG. 25 is a flowchart with exemplary logic for use with the system ofFIG. 19;

FIG. 26A is a simplified block diagram of an exemplary controller foruse with the system of FIG. 19;

FIG. 26B is a flowchart with exemplary logic for use with the controllerof FIG. 26A;

FIG. 27A is a plan view of exemplary parts with exemplary target pointsfor use with the system of FIG. 19;

FIG. 27B is an exemplary machine vision view of exemplary parts for usewith the system of FIG. 19;

FIG. 28A is a perspective view of an exemplary machine vision systemanalyzing an exemplary part with exemplary identified target points foruse with the system of FIG. 19;

FIG. 28B is a plan view of an exemplary optimization analysis of anexemplary part for use with the system of FIG. 28A;

FIG. 29 is a plan view of an exemplary optimization analysis for usewith the system of FIG. 19;

FIG. 30 is a plan view of another exemplary embodiment of an automatedmanufacturing system in accordance with the present invention;

FIG. 31 is a detailed perspective view of a gripper of the system ofFIG. 30 interacting with an exemplary part;

FIG. 32 is a detailed perspective view of the system of FIG. 30 joiningtwo exemplary parts in a fixtureless manner;

FIG. 33 is a perspective view of an exemplary docking station withexemplary locators for use with the system of FIG. 30;

FIG. 34 is a flow chart illustrating exemplary logic for operating thesystem of FIG. 30;

FIG. 35 is a plan view of the system of FIG. 30 operating with the logicof FIG. 34;

FIG. 36 is an exemplary web portal page for use with the system of FIG.30;

FIG. 37 is another exemplary web portal page for use with the system ofFIG. 30;

FIG. 38 is an exemplary comparison report of automated manufacturingusing the system of FIG. 30;

FIG. 39 is an exemplary positional correction report using the system ofFIG. 30;

FIG. 40 is a perspective view of exemplary mobile robot that may beutilized in accordance with the present invention;

FIG. 41A is a detailed perspective view of another exemplary grippingelement that may be utilized in accordance with the present invention;

FIG. 41B is a rear view of an exemplary hybrid fixture assembly,including the gripping elements of FIG. 41A, that may be utilized inaccordance with the present invention;

FIG. 41C is a side view of the hybrid fixture assembly of FIG. 41B;

FIG. 42 is a rear view of another exemplary work area and system usingthe hybrid fixture assembly of FIGS. 41A-41C in accordance with thepresent invention;

FIG. 43A is a perspective view of an exemplary system for applying orremoving fasteners in accordance with the present invention; and

FIG. 43B is a flow chart with exemplary logic for performing thefastener application or removal of FIG. 43A.

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENT(S)

Various embodiments of the present invention will now be described indetail with reference to the accompanying drawings. In the followingdescription, specific details such as detailed configuration andcomponents are merely provided to assist the overall understanding ofthese embodiments of the present invention. Therefore, it should beapparent to those skilled in the art that various changes andmodifications of the embodiments described herein can be made withoutdeparting from the scope and spirit of the present invention. Inaddition, descriptions of well-known functions and constructions areomitted for clarity and conciseness.

Embodiments of the invention are described herein with reference toillustrations of idealized embodiments (and intermediate structures) ofthe invention. As such, variations from the shapes of the illustrationsas a result, for example, of manufacturing techniques and/or tolerances,are to be expected. Thus, embodiments of the invention should not beconstrued as limited to the particular shapes of regions illustratedherein but are to include deviations in shapes that result, for example,from manufacturing.

FIG. 1 illustrates an exemplary AI driven fixtureless and reconfigurablemanufacturing system 10. The system 10 may comprise a machine visionsystem 12. The machine vision system 12 may comprise, for examplewithout limitation, a Cognex® 3D-A5060 available from Cognex Corporationof Natick, Mass. (https://www.cognex.com/). The machine vision system 12may comprise one or more cameras, lasers, radar, proximity detectors,ultrasonic, photo eyes, some combination thereof, or the like. Anylocation, number, orientation, arrangement, and the like of componentsof the machine vision system 12 is contemplated.

The system 10 may comprise one or more joining robots 14. The joiningrobots 14 may comprise, for example without limitation, a Motorman®MA2010 available from Yaskawa® America, Inc. of Miamisburg, Ohio(https://www.motoman.com/en-us). The joining robots 14 may have amaterial handling end of arm tooling that be configured for movement innine dimensions (degrees of freedom). The joining robots 14 may beconfigured to perform welding, fastening, riveting, connecting, joining,some combination thereof, or like functions. In exemplary embodiments,the joining robots 14 may be configured to selectively receive any oneof a number of joining devices 34. Each joining device 34 may beconfigured to perform one or more particular joining techniques. Forexample, without limitation, a given joining device 34 may comprise awelding torch, a screw driver, a riveter, adhesive gun, some combinationthereof, or other similar connection tool functions. The joining device34 may comprise, for example without limitation, a Power Wave® R450 withwire feed available from Lincoln Electric® Company of Cleveland, Ohio(https://www.lincolnelectric.com/en/). In other exemplary embodiments,such joining devices 34 may be permanently attached to the joiningrobots 14. Although two joining robots 14 are illustrated in FIG. 1, anynumber of joining robots 14 are contemplated of the same or differenttypes. Using the geometry datum setting A.I., allows the joining robots14 and the material handling robots 16 to be arranged in various ways,depending on the complexity of the part assemblies in order to handle acomplete parts family without changing of the physical layout andremoving part dedicated tooling figures, around an assembly area 18.

The system 10 may comprise one or more material handling robots 16. Thematerial handling robots 16 may comprise, for example withoutlimitation, a Motorman® MS210 and/or MH50 available from Yaskawa®America, Inc. of Miamisburg, Ohio (https://www.motoman.com/en-us). Thematerial handling robots 16 may have an end of arm tooling that can beconfigured for movement in 9 dimensions (degrees of freedom). Inexemplary embodiments, the material handling robots 16 may be configuredto handle any one of a number of material handling devices 32. Thematerial handling devices 32 may comprise, for example withoutlimitation, a Schunk® PGN-160 and/or PGN-240 available from Schunk IntecInc. of Morrisville, N.C. (https://schunk.com/us_en/homepage/). Eachmaterial handling device 32 may be configured to grasp any one of anumber of parts. In other exemplary embodiments, such material handlingdevices 32 may be attached to the material handling robots 16. Althoughfour material handling robots 16 are illustrated in FIG. 1, any numberof material handling robots 16 are contemplated of the same or differenttypes. The material handling robots 16 may be arranged around theassembly area 18.

The joining robots 14 and material handling robots 16 may be arranged toperform operations on a subassembly 20 located within the assembly area18. The machine vision system 12 may be positioned to view some or allof the assembly area 18. The machine vision system 12 may be withinsufficient proximity and view of the assembly area 18 to be capable ofoptically scanning the subassembly 20. In exemplary embodiments, thesubassembly 20 may be transported into and out of the assembly and weldareas 18 by way of an AGV 28, though any type of transportation for thesubassembly 20 is contemplated.

One or more bins 22 may be positioned around the assembly area 18. Eachof the bins 22 may be configured to hold one or more types of parts 26.In exemplary embodiments, at least one bin 22 may be placed in reach ofeach of the material handling robots 16, though any number andarrangement of the bins 22 is contemplated. Each of the bins 22, inexemplary embodiments, may be tugged by and material handling AGV (notillustrated here) so that the bin 22 is moveable. For example, withoutlimitation, the material handling AGV may move a given bin 22 to be inreach of a given material handling robot 16 and subsequently move thegiven bin 22 to be in reach of another material handling robot 16.

In exemplary embodiments, the machine vision system 12, each of thematerial handling robots 16, each of the joining robots 14, each of thebins 22, and various components thereof may be in electroniccommunication with a controller 30. Such communication may beaccomplished by wired connections, wireless connections, somecombination thereof, or the like. The controller 30 may comprise one ormore electronic storage devices and one or more processors. Executablelearning A.I. software instructions may be stored at the one or moreelectronic storage devices which when executed by the one or moreprocessors configure the one or more processors to perform the functionsand steps described herein. It is also feasible to install theexecutable learning A.I. in an industrial PC, a camera, as part of robotsoftware directly or indirectly, and other devices that may communicatewith the controller 30 or have a more direct communications with theactuators or manipulator arms (servo slides, or robots).

FIG. 2 illustrates an exemplary logic for operating the and AI drivenfixtureless and reconfigurable manufacturing system 10. Each part 26 fora given subassembly 20 may be obtained. In exemplary embodiments, theparts 26 may be stored in, and obtained from, a respective part bin 22,though multiple types of parts may be stored in a given part bin 22.Each part 26 may be locatable by way of an RFID chip, an opticallyscannable code, an indicator, a datum 40, some combination thereof, orthe like. In exemplary embodiments, each of the material handling robots16 and/or material handling devices 32 may comprise a component of themachine vision system 12 for identifying the part 26 for grasping. Thedatums 40 may be particular features of, or locations on, the parts 26or the subassembly 20 which may be stored to product virtual datums 40.Virtual datums 40 may be stored at the controller 30 that represent thelocation of datums 40 of a properly joined subassembly 20. The learningA.I. algorithms preferably have the ability to handle at least hundredsof feature datum geometry settings at the same time.

A given material handling robot 16 may secure a given material handlingdevice 32 to itself. A given part 26 may be gripped by the materialhandling device 32 secured to the material handling robot 16. Thematerial handling robot 16 may move the part 26 and the materialhandling device 32 to a particular location within the assembly area 18.In exemplary embodiments, the material handling device 32 holding thepart 26 may place that part at a docking station 24. This part may be aprimary part. After positioning on the docking station, using camera andA.I., other parts can be placed relative to the primary part. Thisimproves the subassembly quality due to part-to-part variation andtooling variation due to usage and tear and ware. The docking station 24may be positioned on the floor or on an AGV 28 has been moved into theassembly area 18. Additional parts 26 may be gathered followingsubstantially the same or similar steps as needed to compete thesubassembly 20.

During, or as, all parts 26 are gathered, the machine vision system 12may perform an alignment scan of the subassembly 20. The various parts26 may comprise one or more datums 40 in the form of features of thevarious parts 26, such as but not limited to, locating apertures 19,edges, curved surfaces, protrusions, depressions, some combinationthereof, or the like. The location of the datums 40 may be transmittedto the controller 30 where the learning AI algorithm may reside. The AIcompares the location of the scanned datums 40 with the location ofstored virtual datums 40 to determine a best fit for the parts to createthe subassembly stored at the learning A.I. software. This feedbackinspection information will be used by the AI in the following cycle toimprove the quality and fit of the following subassembly during the nextbuild cycle. The various parts 26 may be adjusted by the materialhandling devices 32 and/or the material handling robots 16 as needed. Ifadjustment is performed, the subassembly 20 may be rescanned and furtheradjustments may be made as needed. The process for determining the bestfit and providing spatial adjustment may be as shown and described inU.S. Pat. No. 10,095,214 issued Oct. 9, 2018 and U.S. Pat. No.10,101,725 issued Oct. 16, 2018, which are hereby incorporated byreference herein in their entireties.

The subassembly 20 may be secured. The subassembly 20 may be securedprior to, while, or after performing the alignment scan. In exemplaryembodiments, the subassembly 20 may be secured by activating brakes onthe material handling devices 32 and/or the material handling robots 16.Alternatively, or additionally, the subassembly 20 may be secured bydeactivating motors on the material handling devices 32 and/or thematerial handling robots 16. Another approach is the utilization ofgripping system that can allow the part to be kinematically held withoutthe use of excessive force or the use of a breaking mechanism.

The position of the datums 40 of the various parts 26 as detected by themachine vision system 12 may be transmitted to the joining robots 14.The various parts 26 of the subassembly 20 may be joined by the joiningrobots 14. The subassembly 20 may be, or may remain, secured duringjoining of the parts 26.

After joining the parts 26 to form a completed subassembly 20, thematerial handling devices 32 may be unsecured. The material handlingdevices 32 may be removed from the assembly area 18 by the materialhandling robots 16. An inspection scan may then be performed by themachine vision system 12 to locate the various datum 40. The location ofthe datums 40 from the inspection scan may be compared against thestored virtual datums 40 and/or the location of the datums 40 from thealignment scan to determine if any discrepancies arise. Such comparisonmay be performed at the controller 30, though any location iscontemplated. Any discrepancies between the inspection scan and thealignment scan may be transmitted to the learning A.I. software by wayof the controller 30. The learning A.I. software may be configured tocompute the adjustment vectors needed to be applied to the parts to inorder for the assembly to comply with the stored virtual datums 40 andto compensate for any discrepancies when producing the next subassembly20 as further described herein. This machine learning process may permitoptimization of the assembly process through multiple productioniterations of a given subassembly 20. The flexibility of the materialhandling robots 16 and material handling devices 32 and the use of thelearning A.I. software may provide the ability to use the same, or asubstantially similar, system to handle and join a number of differentlyshaped, sized, arranged, or the like, parts 26 in a number of differentorientations to produce a number of differently shaped, sized, arranged,or the like, subassemblies 20 or assemblies which may be improvedthrough each manufacturing iteration. Furthermore, the use of materialhandling robots 16 and material handling devices 32 to secure the parts26 may provide a true A.I. driven fixtureless assembly process. Bystoring the virtual datums 40 in the A.I. Software (virtually), the needfor a physical fixture to provide physical datum points may beeliminated or significantly reduced.

FIG. 3 illustrates an exemplary gripping element 50. Each of thematerial handling devices 32 may comprise one or more gripping elements50. Each gripping element 50 may comprise one or more motors 52A-C, suchas but not limited to, servos. Each motor 52A-C may be configured tomove the gripping element 50 in a given dimension (e.g., x, y, or zplane or orientation). Three motors 52A-C may be used such that thegripping element 50 is capable of movement in three dimensions. Eachgripping element 50 may comprise a base 56. Each gripping element 50 maycomprise one or more protrusions 58. The protrusions 58 may extend fromthe base 56.

The base 56 may be substantially cuboid in shape, though any shape iscontemplated. The protrusion 58 may comprise cylindrical and/or conicalshaped sections. The protrusion 58 may comprise a ring-shaped indent 54located near a tip 59 of the protrusion 58. Any size or shape protrusion58 is contemplated including but not limited to, grippers, claws,tweezers, clamps, hooks, suction devices, vacuums, some combinationthereof, or the like. Any number of protrusions 58 may be provided on agiven base 56. The protrusion 58, the base 56, or other portions of thegripping element 50 may comprise one or more datums 40.

FIG. 4 and FIG. 5 illustrate an exemplary material handling device 32interacting with an exemplary part 26 of the exemplary subassembly 20.The material handling device 32 may comprise a number of grippingelements 50. The gripping elements 50 may be connected to one anotherthrough a framework of member 36. The gripping elements 50 may beconfigured to interact with various parts 26 of the subassembly 20. Forexample, without limitation, a given gripping element 50 may be placedwithin a locating aperture 19 on a part 26 of the subassembly 20. Asecond gripping element 50 and/or third gripping element 50 may becompressed against walls or other surfaces of the part 26 of thesubassembly 20 to secure the part 26. In exemplary embodiments, thegripping elements 50 may be moved to various locations along the part 26so as to provide a three-dimensional restraint against movement of thepart 26. Edges of the part 26 may be located within the indent 54,though such is not required.

As illustrated with particular regard to FIG. 5, in exemplaryembodiments a first protrusion 58A of a first gripping element 50A maybe placed within a given locating aperture 19 of a given part 26 while asecond protrusion 58B of a second gripping element 50B and a thirdprotrusion 58C of a third gripping element 50C may be located on thewalls of the part 26 on either side of the given locating aperture 19 tosecure the part 26. The second and third protrusions 58B and 58C maycompress the part 26 to secure the part 26 while the first protrusion58A ensures accurate placement. The first protrusion 58A may be pushedout against one edge of the locating aperture 19 to help secure the part26. Additional sets of gripping elements 50 may likewise be secured atother locating apertures 19 of the part 26. In this way, the part 26 maybe secured so that the material handling robot 16 may move the materialhandling device 32, and thus the part 26.

FIG. 6A through FIG. 6D illustrates multiple material handling devices32A-D interacting with multiple parts 26A-D to create the exemplarysubassembly 20. While four material handling devices 32A-D and parts26A-D are shown to create the exemplary subassembly 20, any number ofmaterial handling devices 32 and parts 26 are contemplated to form anykind of subassembly 20. Alternatively, a final assembly 20 may beformed. Each of the material handling devices 32 may be selectivelymountable to the material handling robots 16. In this way, the variousmaterial handling robots 16 may handle a variety of material handlingdevice 32. Each of the material handling devices 32 may likewise gripone or a number of different types of parts 26. In this way, manydifferent types of parts 26 may be handled to create many differenttypes of subassemblies 20 without the need to change hard pointdedicated line tooling to change.

FIG. 7 illustrates an exemplary docking station 24. The docking station24 may be configured for placement on the ground in the assembly area18. Alternatively, or additionally, the docking station 24 may besecured to an AGV 28.

The docking station 24 may be configured to receive one or more parts 26and/or material handling devices 32. In exemplary embodiments, a firstmaterial handling device 32 holding a first part 26 may be rested atopthe docking station 24. Additional material handling devices 32, eachwith one or more parts 26, may be stacked or otherwise adjoined to thefirst material handling device 32 to create the subassembly 20. Thematerial handling robots 16 may be configured to secure each of thematerial handling devices 32 at given positions within the assembly area18. In other exemplary embodiments, once moved, the material handlingdevices 32 or parts 26 may rest in place such that the material handlingdevices 32 may be removed from the respective material handling robots16. The use of the docking station 24 may assist with locating thesubassembly 20. The docking station 24 may additionally, oralternatively, reduce the number of material handling robots 16required. For example, without limitation, a single material handlingrobot 16 may be configured to grab and locate a number of materialhandling devices 32, each with one or more associated parts 26, andstack such parts 26 or material handling devices 32 on the dockingstation 24. It worth noting the docking station is optional and may notbe used.

FIG. 8 illustrates exemplary datums 40. The datums 40 may be located atoverlaps or adjoining areas between two parts 26D and 26C, though anynumber and location of datums 40 are contemplated. The datums 40 may beconfigured for recognition by the machine vision system 12. In exemplaryembodiments, the datums 40 are ball shaped, though any type, shape, andsize datums 40 are contemplated. Alternatively, or additionally, thedatums 40 may comprise certain features of the parts 26 which arerecognizable by the machine vision system 12.

FIG. 9 illustrates the machine vision system 12 locating the variousdatums 40. The machine vision system 12 may comprise executable softwareinstructions stored on one or more electronic storage devices which whenexecuted by one or more processors configure the machine vision system12 to check for alignment of the various parts 26 of the subassembly 20.If further alignment is needed, the controller 30 may send appropriateinstructions to the material handling robots 16. Another alignment scanmay be performed to re-check alignment. When no further alignment isneeded, the controller 30 may direct the joining robots 14 to beginjoining the parts 26. The machine vision system 12 may transmit thelocation of the datums 40 from the last alignment scan to the controller30 to instruct the joining robots 14. Alternatively, or additionally,such executable software instructions and related commands may be storedat the controller 30 and transmitted to the data store FIG. 12 forcapability and trend analysis for example.

FIG. 10 illustrate the joining robots 14 beginning a weld. The joiningrobot 14 may be configured to begin a weld at a datum 40 and travel aspecified distance in a specified direction, though such is notrequired. For example, without limitation, the joining robot 14 may beconfigured to instead begin a weld a specified distance in a specifieddirection from a datum 40. Any type of welding is contemplated such as,but not limited to, shielded metal arc welding, gas metal arc welding,flux cored arc welding, gas tungsten inert gas welding, some combinationthereof, or the like. The weld may be made over a distance or spotwelding may be provided. While welding is discussed, other forms ofjoining the parts 26 are contemplated such as, but not limited to,adhesion, fastening, riveting, crimping, brazing, soldering, shrinkfitting, some combination thereof of the like and the joining robots 14may be adapted for such purposes. Various joining device 34 may beutilized to perform such various types of joining. More than one type ofjoining may be performed.

FIG. 11 illustrates an exemplary completed subassembly 20. Once thevarious parts 26 are joined, the various material handling device 32 mayrelease their grip on the parts 26. The AGV can then move thesubassembly to the next station in the process, until the assembly isdone. The machine vision system 12 may perform an inspection scan of thecompleted subassembly 20. Any misalignment(s) may be reported to thecontroller 30. The subassembly 20 may be secured to an AGV 28 andremoved from the assembly area 18, though any method of transportationis contemplated. Any size, shape, type, number, or the like of parts 26are contemplated to form any size, shape, type, number, or the likesubassembly 20.

FIG. 12 illustrates an exemplary flow diagram for the reconfigurablemanufacturing system 10. The reconfigurable manufacturing system 10 maybe configured to create any number of type of subassemblies 20 with onlyelectronic reprogramming and minimal, if any, retooling or physicalreconfiguration of the physical components. An initial set up processmay be performed. The initial set up may include programming the AIalgorithms with all virtual datums 40 for each subassembly 20 to beproduced. This replaces the 3-2-1 traditional physical fixturing scheme.The initial set up may further comprise programming the controller 30with commands configured to cause the material handling robots 16 andthe material handling device 32 to secure and move the various parts 26,the machine vision system 12 to perform inspection, alignment, and/oridentification scans, such as to determine the location of the datums40, and the joining robots 14 to join the parts 26. The necessary parts26 to create one or more subassemblies 20 may be provided. Preferably,such parts 26 are loaded in the one or more bins 22.

Each of the parts 26 may be subjected to an inspection scan to determinewhich part to grasp and/or verify that the correct part 26 has beengrasped. Each part 26 may be subjected to an alignment scan such thatthe location of the datums 40 may be determined. The machine visionsystem 12 may transmit the location of such datums 40 to the controller30. The controller 30 may compare the location of the scanned datums 40with predetermined locations of virtual datums 40 to determine a bestfit for the parts 26 to create the subassembly 20 stored at the learningA.I. software. The controller 30 may transmit instructions to thevarious material handling robots 16 and/or material handling devices 32to place the parts into a best fit assembly position. Once the parts 26are assembled into the subassembly 20, the machine vision system 12 mayperform an alignment scan to capture the various datums 40 and verifythat the proper positioning has been achieved. The parts 26 may berealigned as needed and subsequent alignment scans may be performed asneeded. A correction model (learning AI algorithm) may optionally be runto compensate for lessons learned during creation of any priorsubassemblies 20 including weld distortions, over torquing and others.The material handling robots 16 may realign the parts 26 as required.

The joining robots 14 may join the parts 26 to create the subassembly20. The material handling device 32 may be removed and the machinevision system 12 may perform an inspection scan the now joinedsubassembly 20. The learning A.I. software may execute a learning model.The learning model may compare the location of the datums 40 from theinspection scan against the expected position of the datums 40. Theexpected position of the datums 40 may be provided by the prioralignment scan(s), the stored virtual datums 40 for the subassembly 20,and/or inspection scans of previous subassemblies 20. The learning modelmay be configured to compensate for any differences by updating thevirtual datums 40 to compensate for such differences. Such differencesmay arise from, for example without limitation, weld spring back,gravity, deflection, compliance, torquing, riveting, some combinationthereof, or the like. Such corrections may be performed by machinelearning A.I. software. The process for determining the best fit,providing spatial adjustment, and compensating for differences may be asshown and described in U.S. Pat. No. 10,095,214 issued Oct. 9, 2018 andU.S. Pat. No. 10,101,725 issued Oct. 16, 2018, which are herebyincorporated by reference. The next subassembly 20 may begin productionby repeating one or more of the aforementioned steps. The leaning modeland/or the database may be located at the controller 30, though such isnot required.

FIG. 13 through FIG. 18 illustrate another exemplary embodiment of thesystem 100 and related methods for operating the same. Similar elementsmay be numbered similarly but increased by 100 (e.g., 16 to 116). Asshown with particular regard to FIG. 13, a worker 160 or other use maygather one or more parts 126 from various part storage tables 123located within or adjacent to the assembly area 118. Alternatively, oradditionally, such parts 126 may be gathered from bins 22. The worker160 may carry one or more parts 126 in the assembly area 118. In otherexemplary embodiments, such parts 126 may be picked up by materialhandling robots 116. Any number and type of parts 126 are contemplatedto form any number or type of subassemblies 120.

The gathered parts 126 may be placed on a table 125, pallet or conveyor.The table 125 may be located near a center of the assembly area 118,though any location within the assembly area 18 is contemplated.Alternatively, or additionally, the parts 126 may be placed at thedocking station 24. The assembly area 118 may be in view of a machinevision system 112. The machine vision system 112 may comprise one ormore cameras, lasers, radar, proximity detectors, ultrasonic, photoeyes, some combination thereof, or the like. Any location, number,orientation, arrangement, and the like of components of the machinevision system 112 is contemplated.

A safety scanner mounted at the bottom of the the table or anotherlocation in the cell may be configured to detect any workers 160 orother individuals in the assembly area 118. If a worker 160 or otherperson is in the assembly are 118, a controller 130 may be configured toprevent machine handling robots 116 from moving or halt the movement ofany machine handling robots 116 in motion.

As shown with particular regard to FIG. 14, once the assembly area 118is clear of workers 160 or other individuals, the controller 130 maysignal the material handling robots 116 to begin or continue working. Asshown with particular regard to FIG. 15, each of the material handlingrobots 116 may comprise a component of the machine vision system 112which may be configured to perform an identification scan to identifyand/or locate the parts 126 on the table for handling. In this way, thecontroller 130 may determine if the correct part 126 is being picked upas well as the location of such parts 126.

In exemplary embodiments, the component of the machine vision system 112may be mounted to a material handling portion 132 of the materialhandling robot 116, though any number and location of components for themachine vision system 112 are contemplated. The various parts 126 may begrasped by the material handling portions 132 attached to the materialhandling robots 116. The material handling portions 132 may compriseclaws, grippers, one or more gripping elements 150, vacuum systems,suction cups, some combination thereof, or the like. Any kind or type ofmaterial handling portion 132 configured to handle any shape or type ofpart 126 is contemplated. Each of the material handling portions 132 maybe configured to handle a number of differently shaped parts 126.

As shown with particular regard to FIG. 16, each of the materialhandling robots 116 may move each of the parts 126 within view of thesame or another component of the machine vision system 112, which may beconfigured to perform a first or additional identification scan toidentify and/or locate the parts 126 within the assembly area 118. Theparts 126 may be moved into an inspection position as needed.

The machine vision system 112 may be configured to perform an alignmentscan. The alignment scan may confirm the location of the parts 126within the assembly area 118. The identification and/or alignment scansmay be made of various datums 140 on the parts 126. The datums 140 maybe recognizable features of the parts 126. Such features may include,for example without limitation, apertures, edges, curved surfaces,protrusions, depressions, some combination thereof, or the like. Theposition of the datums 140 may be compared against stored, virtualdatums 140 at the learning A.I. software of the controller 130. Thelearning A.I. software of the controller may utilize machine learningA.I. software to determine the best fit for the parts 126 to create thesubassembly 120. The position of the parts 126 may be adjusted as neededto match the virtual datums 140. As shown in FIG. 16, the actualposition of the parts 126 may be slightly different that the desiredposition for assembly and inspection. The ghosted image of the parts 126in FIG. 16 may indicate an exemplary position of the parts 126 inaccordance with the virtual datums 140.

As shown with particular regard to FIG. 17, the material handling robots116 may move the parts 126 into an assembly position to form asubassembly 120. The machine vision system 112 may perform an inspectionscan of the subassembly. The inspection scan may be made of the variousdatums 140 on the parts 126. The location, number, and type of datums140 illustrated in FIG. 17 is merely exemplary and is not intended to belimiting. Any number, type, and location of datums 140 for performingany kind of inspection scan and related measurements or determinationsis contemplated. The controller 130 may be configured to determine ifthe location of the datums 140 in the inspection scan matches thevirtual datums 140. If a match is made, or is within a predeterminedmargin of error, the parts may be accepted. If a match is not made, orthe results are outside of a predetermined margin of error, the partsmay be rejected. If a part is accepted, it may be joined. The processfor determining the best fit and providing spatial adjustment may be asshown and described in U.S. Pat. No. 10,095,214 issued Oct. 9, 2018 andU.S. Pat. No. 10,101,725 issued Oct. 16, 2018, which are herebyincorporated by reference. If the part is rejected, the discrepanciesmay be transmitted to a learning module which may utilize the learningA.I. software to compensate for such discrepancies by adjusting thelocation for the virtual datums 140 for the subassemblies 120. The useof material handling robots 116 and material handling devices 132 tosecure the parts 126 may provide a fixtureless assembly process. Bystoring the virtual datums 140, the need for a physical fixture toprovide physical datum points may be eliminated.

In exemplary embodiments, all tasks may be sequenced using Common ObjectRequest Broker Architecture (COBRA) to interact with COBRA actions,though any type or kind of programming language is contemplated. Whilesubassemblies are discussed, it is contemplated that such subassembliesmay be final assemblies as well.

FIG. 19 illustrates an exemplary system 210 for pickup up and placementof parts 236 to form assemblies or subassemblies 220. The same orsimilar components may be numbered similarly but increased by multiplesof 100 (e.g., 12 to 120, 220, etc.). The pickup and placement system 210may comprise a controller 230. The controller 230 may comprise one ormore subcomponents, engines, routines, algorithms, electronic storagedevices, processors, combinations thereof, or the like including but notlimited to, a learning software module 231, a manipulator motion controlengine 233, and/or a power motion controller 235. The system 210 may becomprise one or more materials handling robots 216 and/or machine visionsystems 212. The system 210 may be configured to cause said materialshandling robots 216 to move one or more parts 236 from a first location239 to a second location 241, such as but not limited to, to form one ormore assemblies or subassemblies 220. The machine vision system 212 maycomprise one or more cameras, lasers, range finders, proximity sensors,combinations thereof, or the like located at one or more centrallocations and/or at each material handling robot 216. The machine visionsystem 212 may further comprise machine vision software.

FIG. 20 illustrates an exemplary such material handling robot 216 movingan exemplary workpiece 236 from an exemplary first location 239 to anexemplary second location 241. Upon placement at the second location241, the machine vision system 212 may determine that the workpiece's236 actual position 241B varies from an idea position 241A asillustrated.

FIG. 21 illustrates that multiple materials handling robots 216 and/orjoining robots 214 or other types of robots may be placed in electroniccommunication with one another and/or the controller 230 by way of anetwork 243. In other exemplary embodiments, only certain ones of thematerials handling robots 216 and/or joining robots 214 or other typesof robots may be placed in electronic communication with the controller230 to form the system 210. The system 210 may, alternatively oradditionally, comprise multiple machine vision systems 212 or controller230 linked by one or more networks 243.

FIG. 22 illustrates an exemplary controls interface 245 for the system210.

FIG. 23A through FIG. 23F illustrate various exemplary gripping elements250 and/or components for the same for use with the materials handlingrobots 216. Such gripping elements 250 may, alternatively oradditionally, be utilized as, or in conjunction with, docking devices224 for the subassembly or assembly 220. Compliance devices 281, thepurpose of which will be explained in greater detail herein, may beutilized in connection with the gripping elements 250. Bellows or otherexpansion members 283 may be utilized to adjust the size of the grippingelements 250. Any size or type of gripping element 250 may be utilizedincluding but not limited to, jaws and/or pins. Each gripping element250 may comprise one or more motors, such as servo motors.

FIG. 24 illustrates various exemplary parts 236 which may be manipulatedby the system 210 to create various subassemblies or assemblies 220.

FIG. 25 through FIG. 29 illustrates exemplary logic and relatedcomponents for the system 210 to move parts 236 from the first location239 to the second location 241. The controller 230 may comprise one ormore processors 253 and/or electronic storage device 255. An assemblytable 257, workpiece table 259, and/or target point table 261 may bestored in the memory 255. A learning software algorithm 263 may bestored in the memory 255.

The workpiece table 259 may comprise a list of parts 236, such as bypart identifier, required to create a finished subassembly or assembly220. The assembly table 257 may comprise a list of subassemblies orassemblies 220, such as by assembly identifier or subassemblyidentifier, actually or planned to be created by the system 210. Thetarget point table 261 may comprise one or more target points 240 foreach part 236. Such target points 240 may include, but are notnecessarily limited to, target points 240 for the first location 239and/or the second location 241. In this way, the workpiece table 259 mayprovide a link between the parts 236 to be picked up and the finishedsubassembly or assembly 220 to be created. The target points table 261may comprise one or more target points 240 for each part 259, thusproviding a link between the parts 236 to be formed into the subassemblyor assembly 220.

The controller 230 may be configured to initially populate the targetpoint table 261 with actual measurement data which may be obtained usingthe machine vision system 212. Scans may be made of one or more parts236 and/or final locations for such parts to provide said actual targetpoints 240. In exemplary embodiments, such target points 240 maycomprise coordinates for actual or virtual datums on the part(s) 236such as but not limited to points, edges, holes, surface features,combinations thereof, or the like. Such target points 240 may beextracted by the machine vision system 212 using machine vision or imageanalysis and the locations of these relevant target points 240 may bestored as measured coordinates in the target point table 261. Thesetarget points 240 may be expressed with respect to a reference frameassociated with the workpiece 236 being measured (user frame).Alternatively, if desired, the target points 240 may be expressed withrespect to a reference frame associated with the workstation's dockingstation 224, the materials handling robot 216, the gripping element 250,a joining robot 214, other surface, location, or the like.

In exemplary embodiments, the desired target points 240 may be developedfrom scanning a reference of idealized part 236. In other exemplaryembodiment, such target points 240 may be pre-programmed. Multipletarget points or datums 240 may be developed.

The target point table 261 may comprise a weighting value. The weightingvalue may be assigned for some or all of the target points 240 in thetarget point table 261. The weighting values may be utilized by theoptimization algorithm 263 stored at the controller 230 to control whichtarget point-to-target point relationships need to be relatively tightlyconstrained, and which can be relatively relaxed. By allowing weightedcontrol over which relationships dominate the pickup, the controller 230may calculate an optimal pick-up solution that respects the designengineers' overall vision for the picked-up article 220. In this way,various size and shape parts 236, such as but not limited to square,round, triangle, hat channels, long, short and any other shapes andsizes, may be picked up. The weighting may be utilized to prioritizewhich measured target points 240 to desired target points 240 need to bethe closest to one another. The weighting values may not be a simpleweighted average. Instead, the weighting values operate more as aranking system.

An optimized location of the target points 240 may include positioningthe workpiece 236 in a manner which minimizes the effect that the targetpoints 240 of two or more workpieces 236 have on the magnitude of thevariation in the fabrication of the overall subassembly or assembly 220.The optimized location of the target points 240 is not necessarily thelocation that minimizes the variation between the nominal location ofeach target point 240 and the actual location of each target point 240,as in a least square's regression analysis. The several target points240 may have differing levels of influence on the magnitude of variationin the pickup process of the article 220. Thus, the controller 230 maybe configured to employ a prioritization technique using the weightedvalues by way of the A.I. optimizing algorithm 263. In this way, theseveral target points 240 may be prioritized in the optimizationalgorithm such that the target point or target points 240 that mostinfluence the magnitude of variation in the workpiece 236 can beoriented as close as possible to their nominal, desired target pointpositions to thereby reduce the magnitude of variation in the article220. For example, without limitation, some tolerances may be somewhatarbitrary and that an out-of-tolerance situation for one target point240 does not necessarily render the article 220 defective orinoperative. However, limits may optionally be placed on theoptimization algorithm 263 that would not permit the location of one ormore target points 240 to be positioned at an out-of-tolerance positionwhich could lead to no pick of the part 236.

The target point table 261 may further comprise calculated coordinatesfor each of the target points 240. Initially, these calculatedcoordinate data target points 240 may be unpopulated. The optimizationalgorithm 263 may be configured to use these calculated coordinatesstorage locations to store the intermediate and ultimately the finalcalculated values where each of the target points 240 need to be in thefinal optimized pick-up solution.

The target point table 261 may comprise all target points 240 that arepertinent to the workpiece 236. These target points 240 may include alldatums that need to be used to properly orient and/or otherwise operatethe gripping elements 250. These target points 240 may include allrobots 214, 216, locator holes, and other locator surfaces that are usedto line up the workpiece 236 with the gripping elements 250 (e.g.,mating jaws) found on the docking station 224. Thus, in addition toworkpiece 236 target point data, the target point table 261 may alsocomprise target point location data of the docking station 224 or otherreference point(s) used during the pickup process. This may include thelocation of all gripping elements 250, or other locator structures foundon the docking station 224.

If not already expressed relative to a common reference system, the A.I.optimization algorithm 263 may be configured to perform any necessarycoordinate translation so that all coordinates are expressed relative tothe common reference system, such as but not limited to, the referencesystem of the docking station 224, material handling robot 216, grippingelement 250, or the like. In this regard, one gripping element 250 orother point within the docking station 224 or material handling robot216 may be designated as the primary locator. This primary locator maybe held stationary (i.e., not adjusted by the processor controlledlinear motors) and may serve as the origin point (0,0,0) of the commonreference system, though such is not required.

The processor 253 may be configured to accesses the data structureswithin memory 255, including the tables 257, 259, 261, and may beconfigured to execute the A.I. optimizing algorithm 263. The optimizingalgorithm 263 may include at least the following steps which areprovided for example, without limitation, and which may be repeated,applied in any order, omitted, and/or added to:

1. Designate an origin of a common reference frame, such as about thedocking station 224, the gripping element 250, material handling robot216, or the like, and store this location in the target point table 261as the origin (e.g., 0,0,0). The reference frame may be in a differentposition. For example, the global coordinate system for a car assembly220 may be placed outside the boundary of the docking station 224, workarea, gripping element 250, material handling robot 216, or the like. Insuch a case, the algorithm 263 may be configured to map a transformationmatrix of the coordinates into the common docking station 224, grippingelement 250, material handling robot 216, or the like.

2. Represent the coordinates of the docking station 224, grippingelement 250, material handling robot 216, or the like as variables (tobe determined) in the target point table 261.

3. Express all measured coordinates in the common reference frame.Preferably this common reference frame is that of the workstationdocking station 224, docking station 224, gripping element 250, materialhandling robot 216, or the like, though such is not required. This maybe accomplished by either acquiring the measured coordinates using themachine vision system 212 that is calibrated to the common referencesystem. Alternatively, or additionally, this may be accomplished byperforming matrix transformation of the measured coordinates inworkpiece reference frame into the common reference frame.

4. Using the connected-relationship data stored in the assembly table257 (e.g., ingested from the CAD or other computer aided design orassembly software data), construct a matrix of ordered pairs torepresent each pair of mating target points 240 for the parts 236 of thesubassembly or assembly 220 and store as initial calculated coordinatesin the target point table 261. In the constructed matrix, eachcalculated coordinate may be represented by a test vector of yet to bedetermined length and direction. The vector may have its head coincidentwith one of the mating target points 240 and its tail on the other ofthe mating target points 240.

5. Through an optimization process using priority-based numericaloptimization that takes the stored weighting values into account,computationally iterate through one or more iterations tocomputationally seek the best fit between the measured target points 240and the desired target points 240 for each calculated coordinate pair,giving precedence to target points 240 assigned a relatively higherweight, such that the lengths of all test vectors are minimized but thattest vectors for relatively higher weighted pairs are prioritized to beshortened in length. This algorithm may not be an averaging algorithm.Instead, the algorithm may be configured to prioritize relatively higherweighted target points 240 to reflect their importance to the buildprocess and the end-user. In this way, the algorithm may utilize theweighting values more as a ranking system. The algorithm may utilizelearning, artificial intelligence type techniques. The algorithm may benon-linear multi-object algorithm. The algorithm can handle hundreds oftarget points 240 across multipole parts in a single optimization. Thealgorithm may be built with C++ and may run on a DOS or Linux operatingsystem industrial PC. The algorithm may take input data from processor253, including reference and measured data for the all the target points240 in the target point table 261 and compute a robotic path for thematerial handling robots 216 that allows the pickup and placement system210 to not only find and pick the part 236 correctly, but also placesthe part 236 in its geometrically designed tolerance range, ready forjoining such as by way of the joining robots 214.

6. When the best fit is found, solve for the positions of all materialhandling robots 216, docking stations 224, and/or gripping elements 250.

7. Use the calculated positions of the moveable locators and positioningstructures to drive the linear motors to adjust the material handlingrobots 216 and/or gripping elements 250 in physical space. For example,a scan may be previously been performed of a desired location for thepart 236, or such a location may be preprogrammed.

The optimization algorithm 263 may be configured to utilize test vectorsto seek the best fit, optimal solution. As shown particularly in FIGS.27A-28B, three exemplary target points 240A, 240B, 240C may be selectedon an exemplary workpiece 236. The three target points, referred to inthe figures as Target Point 1, 2, 3, may be selected in referencenominal space, with their respective measured target points 240A, 240B,240C (in this case represented by small circles) placed approximately oncorner edge points. Between each pair of mating target points 240, atest vector 271 may be defined in virtual space within the controller230. Because each workpiece 236 is represented in the common coordinateframe, each of these test vectors 271 may reference the origin point ofthe coordinate system. By way of example, without limitation, we shallassume that workpiece 236 is oriented on a pallet in an actual position273 that differs from the designed position 275 (nominal or referenceposition). Each target point 240 remains of unknown position until theoptimization algorithm 263 is run. This includes the position of alltarget points 240, which corresponds to actual measured data. When theoptimization algorithm 263 is run, a solution may be calculated whichminimizes the length of all of the test vectors 271 (e.g., by utilizingthe 3-dimensional measured data and ideal nominal reference data),taking individual weighting of each pair of mating target points 240into account. The effect of the algorithm 263 may be to determine thefinal location of all target points 240 in the optimal arrangement. Thishas been diagrammatically illustrated in at least FIG. 29 which showsthe condition of three exemplary test vectors 271 a, b and c before anyoptimization is performed at 277. The test vectors 271 each have adirection and length sufficient to connect its respective pair of matingpoints. Because no optimization has been performed at this stage, thesetest vectors may be of any length and of any direction.

As the priority-based numerical optimization algorithm 263 is run, suchas but not limited to iteratively or recursively a number of times, theoptimal lengths and directions of test vectors 271 a, b and c may beultimately arrived at as shown at 279. In this example, test vector 271b received highest weighting priority, resulting in its associated pairof begin and end points being coincident. The other test vectors 271 aand c have been shortened in length, although not as much as vector b.Note that for this example, which is provided without limitation, testvectors 271 a and c have received adjustment in pointing direction aswell.

Essentially, the optimized change in length and direction of the testvectors 271 may correspond with a shift in the three-dimensionalposition of the workpieces 236 to which the corresponding target points240 are associated. Once the optimized solution 279 is achieved, theoptimized positions of all target points 240 may be fixed in referencecoordinate space. That is, the position of each target point 240 may bedetermined by first establishing the locations relative to the origin(0,0,0) of the workpiece 236. Then the locations of the contact pointsfor pickup may be determined by minimizing the test vectors 271, usingthe optimized test vectors adjustments which then can be transformedusing Euler angles transformations to robot coordinates.

As noted above, once the best fit 279 for the workpieces 236 has beendetermined in virtual space, the controller 230 may be configured tosolve for the required coordinates of the material handling robots 216and/or gripper elements 250 and automatically move to proper pickuplocations.

Having thus presented an explanation of the optimization algorithm 263,use of the overall system 210 may now be discussed.

Joining Workpieces Using Variable Position Locators:

An exemplary method for performing an A.I. pick and place operation at agiven workstation may begin with calibration of the machine visionsystem 212. Once calibrated, the parts 236 to be picked up may beidentified using the machine vision system 212. The geometry and shapeof the parts 236 may vary without substantially affecting the method foridentifying target points 240 and orienting the materials handling robot216 to actual workpiece 236 configuration.

The machine vision system 212 may collect and analyze 3-dimensional dataregarding selected targets points on a workpiece 236. In the exampleshown, without limitation, which depicts a box 236 to be picked up, thetarget points comprise 3-targets 240A, B, C. Each target point 240 maybe established as a reference position based on a design range.

The controller 230 may determine, based on a scan of the part 236 by themachine vision system 212, if the several target points 240 are in theiroptimized location, such as within a predetermined range of desiredtarget points 240. If so, the controller 230 may program the robots 214to pick up the workpiece 236 as it normally would. If not, theoptimization algorithm 263 may be performed. Specifically, if the targetpoints 240 are not in their optimized locations, the gripping elements250 or other components of the material handling robots 214 may be movedby motors, such as but not limited to linear servo motors, as requiredto position part 236 such that the target points 240 on the workpieces236 are at the optimized location 279 for pickup. The system 210 mayconfirm the positioning of the target points 240 in their optimizedlocations 279 (such as within predefined limits) by way of a second scanby the machine vision system 212 and to permit the controller 230 toconfirm that the optimized locations have not changed. The desiredtarget points 240 may be developed from scans of an idealized referencepart 236 and/or the location where such parts 236 are to be placed.

The system 210 may be configured to adjust the position of parts 236relative to one another to fit within an overall design scheme whileminimizing the amount of adjustment that is needed to each individualpart 236. This is particular helpful as parts may be shifted inplacement when presented to materials handling robots 214 and/or whengripped by gripping elements 250 or may be provided in varying sizesand/or shapes. While such shifts and/or variations may be small, theymay lead to larger gaps in the overall subassembly or assembly 220,leading to a defective article 220. The controller 230 may utilize amachine vision system 212 to identify parts for pickup and placement toform an assembly 220 based on parts 236 assigned to an assembly 220 aslaid out in the workpiece table 259. Target points 240, which maycomprise datums, for each part 236 may be determined from a target pointtable 261 and/or initially determine from scans of reference versions ofthe parts 236 and/or the part's 236 desired location by the machinevision system 212. Each actual measured target point 240 may be matchedwith a desired target point 240 to form a coordinate pair. For example,without limitation, some coordinate pairs may be related to a pick-uplocation and other may be related to a placement location.

The controller 230 may be configured to run a multi-iterative best fitprocess to determine, using the test vectors 271, and accounting forweighting assigned to the target points 240 at the target point table261, the minimum amount of movement of each part 236 needed to create anin-tolerance article 220. Once a best fit solution 279 is found, it maybe executed and a confirmation scan by the machine vision system 212 mayoptionally be performed and further adjustments may be made as needed.

Relatively higher weighted target points 240 may be prioritized. Forexample, without limitation, while the solution having the overallshortest length summed vectors may be desired (reflecting overall lowestamount of movement), a slightly longer overall length summed vectorssolution may instead be selected where the vector associated with therelatively highest weight is the shortest compared to other potentialsolutions. In this way, relatively close tolerance parts (which may beassigned higher weights) may be made more likely to stay in tolerance bypermitting less shifting between their actual position and their desiredposition. Likewise, relatively loose tolerance parts (which may beassigned lower weights) may be permitted greater latitude in shifting,as such movement is unlikely to result in an out of tolerance article220.

In exemplary embodiments, the system 210 may be configured to provide atleast the following movements: positioning, holding, immobilizing, andinterfacing, each of which is further described herein.Positioning—precisely aligning and locating the part using the algorithm263 and robot 216 to place a part 236 within its design tolerance range.Holding—eliminating the degrees of freedom of the part 236 with respectto the robot 216, docking station 224, or the like to hold it ingeometrical constrained status. Immobilizing—resisting movement and/ordeflections of the part 236 against the forces of the manufacturingoperations. The following types of immobilizing are provided asnon-limiting examples: resist forces from contacting part duringjoining; and resist forces from the joining equipment such as joiningrobots 214. Interfacing—allowing interfacing and avoiding interferencebetween various pairs of agents during manufacturing operations. Thefollowing types of interfacing are provided by way of non-limitingexample: part-to-part (avoid gripping elements 250 from getting inbetween parts 236 joined); and part-to-tool (avoid gripping elements 250from obstructing access of the joining robots 214).

Exemplary system 210 specifications include, without limitation, theability to move parts weighing 2001 b and smaller, a stroke range of 500mm (250 mm of travel per jaw), non-slippage, and one or more compliancedevices 281 in center of actuator of each gripping element 250 tostabilize large flexible material parts 14 mm of compression on z-axis.The system 210 may then travel the robot 216 into the part 236,compressing that spring to a certain depth, then sensors may detect theZ stroke and the controller 230 may register the compression and send acommand to the servo to close the jaws 250. Use of a machine visionsystem 212 to acquire target point data and present to the algorithm 263to provide accurate position to identify to the controller 230 theposition of the part 236 to a tight accuracy and so that the robots 216may be best positioned before closing the gripper 250.

FIG. 30 through FIG. 39 illustrate another exemplary embodiment of thesystem 310 and related methods for operating the same. Similar elementsmay be numbered similarly but increased by 100 (e.g., 16 to 116, 216, or316). The controller 330 may comprise multiple subcomponents, at leastsome of which may be housed within one or more storage boxes 385, suchas electrical boxes. In exemplary embodiments, the controller 330 maycomprise an industrial PC 330A or other computing device(s). Theindustrial PC 330A may comprise, for example without limitation, a Boxer6405 available from Aaeon® Technology Inc. of Hazlet, N.J.(https://www.aaeon.com/en/). The controller 330 may comprise a PLC 330Bor other controller(s). The PLC 330B may comprise, for example withoutlimitation, a CompactLogix™ 5370 available from Rockwell Automation®,Inc. of Milwaukee, Wis. (https://www.rockwellautomation.com/en-us.html).The PLC 330B in exemplary embodiments, may be electronically interposedbetween some or all of the industrial PC 330A, the machine vision system312, each of the joining robots 314, each of the material handlingrobots 316, and components of the same, such as show in FIG. 35 forexample without limitation. One or more user devices 365 may beelectronically connected to the PLC 330B and/or the industrial PC 330Aand/or other component of the controller 330. Such electronicinterposition and/or connection may be accomplished by hardware and/orsoftware. For example, without limitation, the controller 330 maycomprise an ethernet switch 369 which provides network communicationbetween such components. Alternatively, or additionally, the controller330 may be configured to recognize commands or other data fromregistered user devices 365 or users thereof, such as by login or otherauthentication techniques. One or more power supplies 371 may beprovided, such as within the storage compartment 385 to power one ormore of such components.

In other exemplary embodiments, the controller 330 and various relatedequipment, including but not limited to, the power supplies 371,industrial PC 330A, PLC 330B, and/or ethernet switch 369 may beintegrated with one or more of the robots, including but not limited to,the joining robots 14, material handling robots 16, AVGs 28,combinations thereof, or the like. Such components may be integratedwith the machine vision system 112, alternatively or additionally. Themachine vision systems 112 may alternatively or additionally be sointegrated with such one or more robots.

Communication between the controller 330 and the user device 365 may beaccomplished by way of one or more internet portals 367, for examplewithout limitation. The user device 365 may comprise one or morepersonal computers, tablets, servers, smartphones, combinations thereof,or the like. The internet portal 367 may provide one or more pagesconfigured to accept user input for programming the system 310 and/ormonitoring operations of the system 310. FIG. 36 provides an exemplarypage 367A of the internet portal 367 for accepting user input regardingnominal data measurements or other data for one or more parts 326A-Dand/or one or more subassemblies 320 to be generated. Such user inputmay include, for example without limitation, where such parts 336 shouldbe joined, such as by welding, to form part of all of the subassembly320. Such user input may alternatively or additionally include, forexample without limitation, information such as material type, type ofjoining to be performed, weld information (thickness, length, etc.),combinations thereof, or the like. The user input may be used by thesystem 310 to provide automated manufacture of the subassembly 320. FIG.37 provides an exemplary page 367B of the internet portal 367 providingan example of monitoring operations for the system 310, such as throughvarious statistics and/or charts, such as but not limited to one or moretrend charts. The controller 330 may be configured to provideinformation or other data sufficient to generate such statistics and/orcharts. FIGS. 38 and 39 provide exemplary repots 391A and 391B ofoperation of the system 310 with exemplary parts 326 to create exemplarysubassemblies 320.

FIG. 33 illustrates an exemplary docking station 324 for the part(s) 326and/or subassembly 320. The docking station 324 may comprise one or moresupports 321, which may be static or dynamic. The supports 321 may beconfigured to support and/or temporarily secure the part(s) 326 and/orsubassembly 320, such as in one or more elevated positions to permitmanipulation and/or joining of such part(s) 326 and/or subassembly 320.In exemplary embodiments, the docking station 324 may comprise one ormore of the tables 257. Some or all of the supports 321 may comprise oneor more material handling device 332 and/or gripping elements 350. Someor all of these material handling devices 332 and/or gripping elements350 may act as fixed or adjustable locators. Other fixed and/oradjustable locations may be provided. For example, without limitation,such locators may be electronically connected to the controller 330,wired and/or wirelessly, to take reference location measurements of thepart(s) 326 and/or subassembly 320. Such locations may alternatively oradditionally be independent of the docking station 324.

As provided with particular regard to at least FIGS. 34-35, the materialhandling robots 316 may be commanded by the controller 330 to pick upone or more parts 326 and move them within visual range of the machinevision system 312 for identification, dimensional measurement, and/orposition measurement. Such movement may place the parts 326 withinvisual range and/or move the parts 326 within a visual range into apreferred position for inspection. Alternatively, or additionally, suchparts 326 may be placed within a docking station 324 and additionalmeasurement data may be retrieved from locators associated with thedocking station 324. The controller 330 may be configured to compare thedata from the machine vision system 312 to nominal part data, such asinputted at the user device 365. The controller 330 may be configured tocommand the material handling robots 316 to perform one or more positioncorrections of the parts 326 and/or subassemblies 320 to account foroffsets or other discrepancies between the nominal part data and theactual measurements. In exemplary embodiments, without limitation, thecontroller 330 may be configured to utilize a vector and priority basednumerical optimization program. The controller 330 may be configured tocommand the material handling robots 316 to position the parts 326relative to one another to form some or all of the subassembly 320 inaccordance with the nominal part data as corrected by the controller 330to account for the offsets or other discrepancies. The controller 330may be configured to subsequently command material joining robots 314 tojoin certain portions of the parts 326, such as but not limited to, atthe areas identified in the user input. In other exemplary embodiments,the areas for joining may be automatically determined by the controller330 based on the user input describing the subassembly 320 to bemanufactured.

One or more of the material handling robots 316 may be configured tomove the joined parts 326 in view of, and/or into a preferred inspectionposition, the machine vision system 312 for inspection of the joinedsubassembly 320. An inspection scan may be performed by the machinevision system 312 and the data may be transmitted to the controller 330for comparison against user input, nominal data, and/or expected data.The controller 330 may be configured to utilize one or more machinelearning algorithms to update the vector and priority based numericaloptimization program in accordance with this received data. For example,without limitation, if the inspection scan reveals that a significantmargin exists between a particular nominal point and an actuallymeasured point, the controller 330 may be configured to provide a higherweighting to this nominal point in the vector and priority basednumerical optimization program when building the next subassembly 320.In this matter, subsequent subassemblies 320 may be manufactured ingreater compliance with the nominal data. As another example, withoutlimitation, if the inspection scan reveals that a relatively lower thanneeded margin exists between a particular nominal point and an actuallymeasured point, the controller 330 may be configured to provide a lowerweighting to this nominal point in the vector and priority basednumerical optimization program when building the next subassembly 320.In this manner, subsequent assemblies 320 may be manufactured withincompliance with the nominal data and freeing other portions of the parts326 for greater compliance or tighter tolerancing with the nominal data.This is just one example, other learning mechanisms, including thoseusing the same or other artificial intelligence techniques, may beutilized by the controller 330.

The techniques shown and/or described herein, and particularly the Alsystems, are not limited to positional geometry (such as but not limitedto, three-dimensional position coordinates (X, Y, Z), pitch, yaw), butmay also detect, process, and account for other data from additionalsensors such as, but not limited to, vibration, force, speed, and canadjust operations to accommodate changes necessary to adapt to suchfactors. Such techniques may be utilized for actual manufacturing orsimulated representations of a manufacturing process. Any number ofiterations may be performed to improve simulated or actual manufacturingprocesses based on iterative feedback of any number or kind ofdatapoints.

FIG. 40 illustrates an exemplary mobile robot system 500. Welding and/orjoining robots 14, 16 may be mounted to AGVs 28. The AGVs 28 maycomprise one or more wheels 504 or other components which providemobility for the AGVs 28. One or more tool changers 502 may be mountedto the AGVs 28. Some or all components of the systems 10, 100, 210, 310such as but not limited to the controller 330, may be mounted to theAGVs 28. Some or all components of the machine vision system 112 may bemounted to the robots 14, 16 and/or the AGVs 28.

FIG. 41A illustrates exemplary gripping elements 550. FIG. 41B and FIG.41C illustrate an exemplary AGV system 528 incorporating the grippingelements 550 as part of a hybrid fixture subassembly 524. The grippingelements 550 may be configured to provide three-dimensional movement toflexibly adapt to, or adjust, part 526A, 526B location, such as to closegaps between parts to conform to nominal measurement data and joining.Such gripping elements 550 may be used in conjunction with rigid ormoveable fixture elements (e.g., members, framework, surfaces,combinations thereof, or the like) used to position such grippingelements 550 or provide surfaces for material handling and/or placement.

Portions of the hybrid fixture subassembly 524 may be detachable. Forexample, a first portion of the hybrid fixture subassembly 524 mayremain with the AGV 528 while a second portion may be moved by one ormore machine handing robots 16. In this manner, the parts 526 may bepositioned within the first portion of the hybrid fixture subassembly524 while moved. The second portion of the hybrid fixture subassembly524 may be permanents, semi-permanently, or detachably secured to theAGV 28, for example.

Portion of the hybrid fixture subassembly 524, other than the grippingelements 550, may be position adjustable in exemplary embodiments. Thismay permit movement of the hybrid fixture subassembly 524 and/or partssecured therein. Motors or other control elements may be commanded bythe controller 330, in exemplary embodiments.

Clamping devices 527 may be provided at the hybrid fixture subassembly524 for selectively clamping or otherwise securing one or more parts 526to the hybrid fixture subassembly 524.

FIG. 42 is a rear view of a system 600 utilizing the hybrid fixturesubassembly 524. One or more machine vision elements 112 may be mountedto the robots 14, 16, the AGV 28, and/or be provided within or adjacentto a work area 618. Multiple such machine vision elements 112 may beused and placed in communication with each other and/or the controller330, for example to verify positions of the parts 526 and/or assembliesas they move through the manufacturing process. The parts 526 may becarried into and/or out of the work area 618 by one or more AGVs 528,such as those incorporating the hybrid fixture subassemblies 524 by wayof non-limiting example.

FIG. 43A and FIG. 43B illustrate exemplary applications of the systems10, 100, 210, 310, 500, 600 for adding or removing fasteners. Actualpositions of fasteners may be identified on parts 526 and/or assembliesand compared with nominal data. Positioning of robot 14, 16 adapted toadd or remove such fasteners and/or position the parts 526 and/orassemblies may be adjusted and the fasteners, such as but not limited torivets, may be added or removed from a part 526 and/or completedassembly. The robots 14, 16 may comprise one or more specialized toolsfor identifying, adding, or removing such fasteners by way ofnon-limiting example. Any type or kind of fastener may be utilized.Those of skill in the art will recognize that the types and kinds ofapplications shown and/or described herein are merely exemplary and arenot intended to be limiting. The systems and methods shown and/ordescribed herein may be utilized in any number of applicationsincluding, but not limited to, material handling, welding, adhesion,fastening, inspection, unfastening, separation, destruction, qualitychecking, grinding, machining, combinations thereof and the like. Robotsand tools appropriate to such tasks may be utilized.

The systems and methods shown and/or described herein may utilizeartificial intelligence to improve the manufacturing process over time,such as by comparing the finished article with the nominal datameasurements as part of a finished article inspection scan. Adjustmentsmay be made in the next processed article to compensate in variationsbetween nominal data measurements and target data points.

The systems and methods shown and/or described herein may be configuredto autonomously generate and commence manufacturing instructions, suchas at the controller 330, for the various robots 14, 16 to perform basedon a scan of the article to be created. The controller 330 may derivenominal data measurements for target features of the article scanned andcommence automated manufacture of the same. For example, the variousparts 526 may be initially scanned to derive target points of featuresof the parts 526. The controller 330 may be configured to identify theparts 526 and/or determine how they must be manipulated to form thearticle conforming to the nominal data measurements. The controller 330may be configured to command the robots 14, 16 to so operate to form thearticle. Formation may include any type, number, or kind ofmanufacturing step(s) (e.g., joining, moving, inspection, materialremoval, etc.). The controller 330 may be configured to perform a finalinspection scan of the finished article, and such as by utilizing an AIprogram, make adjustments to the next article manufactured. Suchadjustments may be, for example without limitation, made to weighting ofone or more priority based numerical optimization programs.

The systems and methods shown and/or described herein may be used withany number or type of fixturing systems, including but not limited to,entirely fixtureless systems, entirely fixtured systems, and hybridsystems (part fixtured, part fixtureless). Robotic vehicles may beutilized with some or all of the components of the systems, such as butnot limited to, for transporting parts, subassemblies, or assembliesto/from a work area, on or separate from, fixated or fixturelesssystems. The systems shown and/or described herein may be utilized withany type of kind of manufacturing processes, including but not limitedto, material handling and placement, welding, adhesion, fastening,unfastening, inspection, combinations thereof, and the like. Theartificial intelligence adaptation systems and methods shown and/ordescribed herein may be used with any applications shown and/ordescribed herein, such as but not limited to, for fastening tocompensate for over or under torquing, placement, spring back,combinations thereof, or the like. Such AI adaptation may be performedon a priority basis, such as to prioritize particular feature targetpoints by adaptive weightings.

Any embodiment of the present invention may include any of the featuresof the other embodiments of the present invention. The exemplaryembodiments herein disclosed are not intended to be exhaustive or tounnecessarily limit the scope of the invention. The exemplaryembodiments were chosen and described in order to explain the principlesof the present invention so that others skilled in the art may practicethe invention. Having shown and described exemplary embodiments of thepresent invention, those skilled in the art will realize that manyvariations and modifications may be made to the described invention.Many of those variations and modifications will provide the same resultand fall within the spirit of the claimed invention. It is theintention, therefore, to limit the invention only as indicated by thescope of the claims.

Certain operations described herein may be performed by one or moreelectronic devices. Each electronic device may comprise one or moreprocessors, electronic storage devices, executable softwareinstructions, and the like configured to perform the operationsdescribed herein. The electronic devices may be general purposecomputers or specialized computing device. The electronic devices may bepersonal computers, smartphone, tablets, databases, servers, or thelike. The electronic connections and transmissions described herein maybe accomplished by wired or wireless means.

What is claimed is:
 1. A system for automated manufacture of an articlefrom parts, said system comprising: one or more automated materialhandling robots, each having a portion for securing said parts; one ormore machine vision systems positioned to collectively view a work areafor said one or more automated material handling robots; a controller inelectronic communication with each of said one or more automatedmaterial handling robots and said one or more machine vision systems,wherein said controller comprises software instructions stored on one ormore electronic storage devices which when executed, configures one ormore processors of said controller to: obtain nominal data measurementsfor said article; perform an identification scan of said parts within oradjacent to said work area by said one or more machine vision systems;perform an initial scan of said parts within said work area by said oneor more machine vision systems identified as being needed to form saidarticle to identify target points on each of said; compare, at saidcontroller, said target points with said nominal data measurements; andcommand said one or more automated material handling machines to graspand move said parts within said work area to form said article.
 2. Thesystem of claim 1 wherein: said nominal data measurements are obtainedfrom a scan, performed by said one or more machine vision systems, of areference article placed within said work area.
 3. The system of claim 1wherein: said nominal data measurements are obtained from user inputprovided at one or more user systems in electronic communication withsaid controller.
 4. The system of claim 1 further comprising: one ormore autonomous guided vehicles comprising one or more fixturingassemblies configured to hold at least some of said parts in electroniccommunication with said controller.
 5. The system of claim 4 wherein:said one or more fixturing assemblies comprise moveable fixturingelements.
 6. The system of claim 1 further comprising: one or moreautonomous guided vehicles, wherein said automated material handlingrobots are mounted to one or more of said autonomous guided vehicles. 7.The system of claim 6 wherein: said automated material handling robotscomprise one or more tools for removing or adding fasteners, includingrivets.
 8. The system of claim 1 wherein: said one or more electronicstorage devices of said controller comprise an assembly table, aworkpiece table, and a target point table; said assembly table comprisessaid nominal data measurements for said article; said workpiece tablecomprises data regarding said parts required to form said article; andsaid target point table comprises said target points for said parts. 9.The system of claim 1 further comprising: additional softwareinstructions stored on said one or more electronic storage devices ofsaid controller, which when executed, configure said one or moreprocessors of said controller to associate at least some of said nominaldata measurements with a non-zero weighting value in accordance with avector and priority based numerical optimization program.
 10. The systemof claim 9 wherein: said vector and priority based numericaloptimization program is configure to cause said controller to perform aniterative analysis to determine a best fit solution for moving saidparts such that each of said target points measured in said initial scanare within a predetermined range of an associated one of said nominaldata measurements; said best fit solution is determined by fittingvectors between each of said features and said associated one of saidnominal data measurements to develop a solution set; and said best fitsolution is a solution from said solution set having the overallshortest length of summed vectors while prioritizing relatively higherweighted nominal data measurements.
 11. The system of claim 10 wherein:said best fit solution comprises said solution where said vector forsaid distance between said nominal data measurements and said associatedone of said features is smallest for said relatively highest weightedone of said nominal data measurements.
 12. The system of claim 10wherein: said relatively higher weighting values are assigned torelatively close tolerance requirements between certain of said nominaldata measurements of certain of said parts and said relatively lowerweighting values are assigned to relatively lower tolerance requirementsbetween certain of said nominal data measurements for certain of saidparts.
 13. The system of claim 10 wherein: said iterative analysisutilizes artificial intelligence techniques.
 14. The system of claim 13further comprising: additional software instructions stored on said oneor more electronic storage device of said controller, which whenexecuted, configure said one or more processors of said controller to:command said one or more automated material handling robots to move saidarticle in view of said one or more machine vision systems; command saidone or more machine vision systems to perform an inspection scan of saidarticle manufactured by said system to obtain finished article targetpoints.
 15. The system of claim 14 wherein: said artificial intelligencetechniques comprise: identifying discrepancies between said nominal datameasurements and said finished article target points as measured duringsaid inspection scan; and adjusting said weightings of said vector andpriority based numerical optimization program in accordance with saiddiscrepancies.
 16. The system of claim 15 wherein: at least some of saidweightings assigned to with nominal data measurements associated withdiscrepancies above a first threshold are increased; and at least someof said weightings assigned to with nominal data measurements associatedwith discrepancies below a second threshold are decreased.
 17. Thesystem of claim 1 wherein: said controller comprises an industrial PCand a PLC which is electronically interposed between said industrial PC,each of said one or more automated material handling robots, and saidone or more machine vision systems.
 18. A method for automatedmanufacture of an article from parts, said method comprising the stepsof: obtaining nominal data measurements for said article; performing, bya machine vision system, an identification scan of said parts within oradjacent to said work area; causing one or more automated materialhandling robots to move said parts within view of said machine visionsystem; performing an initial scan of said parts machine vision systemto identify target points on each of said parts; comparing, at acontroller in electronic communication with said machine vision systemand said automated material handling robots, said target points withsaid nominal data measurements; and commanding, by way of saidcontroller, said one or more automated material handling machines tograsp and move said parts within said work area to form said article.19. The method of claim 18 wherein: the step of obtaining nominal datameasurements for said article comprises the subsets of performing, by amachine vision system, a scan of a reference article placed within awork area.
 20. The method of claim 18 wherein: the step of obtainingnominal data measurements for said article comprises the subsets ofelectronically communicating, from one or more remote user systems, userinputs for said article.
 21. The method of claim 18 further comprisingthe steps of: associating at least some of said nominal datameasurements with a non-zero weighting value in accordance with a vectorand priority based numerical optimization program by performing aniterative analysis to determine a best fit solution for moving saidparts such that each of said features is within a predetermined range ofan associated one of said nominal data measurements, determining saidbest fit solution by fitting vectors between each of said nominal datameasurements and said associated one of said features to develop asolution set, where said best fit solution is a solution from saidsolution set having the overall shortest length of summed vectors whileprioritizing relatively higher weighted nominal data measurements. 22.The method of claim 21 wherein: said best fit solution comprises saidsolution where said vector for said distance between said nominal datameasurements and said features is smallest for said relatively highestweighted nominal data measurement; and said relatively higher weightingvalues are assigned to ones of nominal data measurements havingrelatively close tolerance requirements and said relatively lowerweighting values are assigned to ones of said nominal data measurementshaving relatively lower tolerance requirements.
 23. The method of claim18 further comprising the steps of: commanding, by way of saidcontroller, an automated material joining machine to weld said parts tocreate said article; causing said one or more automated materialhandling robots to move said welded article in view of said one or moremachine vision systems; performing, by said machine vision system, aninspection scan of said welded article to measure final featurelocations; identifying, at said controller, discrepancies between saidnominal data measurements and final feature locations as measured duringsaid inspection scan of said welded article; and adjusting, at saidcontroller, said weightings of said vector and priority based numericaloptimization program in accordance with said discrepancies.
 24. Themethod of claim 23 further comprising the steps of: revising saidweighting by at least: increasing at least some of said weightingsassigned to with nominal data measurements associated with discrepanciesabove a first threshold; and decreasing at least some of said weightingsassigned to with nominal data measurements associated with discrepanciesbelow a second threshold; and manufacturing an additional one of saidarticles in accordance with said revised weightings.
 25. A system forautomated manufacture of an article from parts, said system comprising:a user system configured to facilitate internet access; one or moreservers in network communication with said user system and configured tohost a portal for receiving user input regarding nominal datameasurements for said article to be formed by said parts; a number ofautomated material handling robots, each having one or more elements formoving individual ones of said parts; one or more automated materialjoining machines, each having a welding tool for joining various areasof said parts to form said article; a machine vision system comprising acamera and positioned to view a work area; a controller in electronicnetwork communication with each of said automated material handlingrobots, said automated material joining machines, said machine visionsystem, and said user system, wherein said controller comprises softwareinstructions stored on one or more electronic storage devices which whenexecuted, configures one or more processors of said controller to:receive said user input from said user system by way of said portal;command each of said number of said automated material handling robotsto move a respective one of said number of parts within view of saidmachine vision system for identification and inspection; command saidmachine vision system to perform an initial scan of said parts toidentify each of said parts and determine dimensional measurementsbetween or of features of each of said parts; associate each of saidnominal data measurements with a non-zero weighting value in accordancewith said user input, wherein relatively higher weights are assigned tonominal data measurements indicating critical dimensions or tolerances;determine discrepancies between said nominal data measurements and saiddimensional measurements from said initial scan; perform an iterativeanalysis to determine a best fit solution for moving said number ofparts relative to another such that said features are within apredetermined range of an associated one of said nominal datameasurements, wherein said best fit solution is determined by fittingvectors between each of said features and said associated one of saidnominal data measurements to develop a solution set, and said best fitsolution is a solution from said solution set having the overallshortest length of summed vectors while prioritizing relatively higherweighted nominal data measurements; command each respective one of saidnumber of automated material handling robots to move each respective oneof said number of parts into positions relative to one another inaccordance with said best fit solution to form at least part of saidarticle; and command said one or more automated material joiningmachines to join said number of parts at said areas to form saidarticle.